Human unintentional and intentional interpersonal coordination in interaction with a humanoid robot.

In order to establish natural social synchrony between two humans, two requirements need to be fulfilled. First, the coupling must be bi-directional. The two humans react to each other's actions. Second, natural social bodily synchronization has to be intentional or unintentional. Assuming that these essential aspects of human-human interactions are present, the present paper investigates whether similar bodily synchrony emerges between an interacting human and an artificial agent such as a robot. More precisely, we investigate whether the same human unintentional rhythmic entrainment and synchronization is present in Human Robot Interaction (HRI). We also evaluate which model (e.g., an adaptive vs non adaptive robot) better reproduces such unintentional entrainment. And finally, we compare interagent coordination stability of the HRI under 1) unidirectional (robot with fixed frequency) versus bidirectional (robot with adaptive frequency) rhythmic entrainment and 2) human intentional versus unintentional coupling. Fifteen young adults made vertical arm movements in front of the NAO robot under five different conditions of intentional/unintentional and unidirectional/bidirectional interactions. Consistent with prior research investigating human-human interpersonal coordination, when humans interact with our robot, (i) unintentional entrainment was present, (ii) bi-directional coupling produced more stable in-phase un-intentional and intentional coordination, (iii) and intentional coordination was more stable than unintentional coordination. To conclude, this study provides a foundation for modeling future social robots involving unintentional and bidirectional synchronization-aspects which seem to enhance humans' willingness to interact with robots.

Socio-Motor Improvisation in Schizophrenia: A Case-Control Study in a Sample of Stable Patients.

Improvising is essential for human development and is one of the most important characteristics of being human. However, how mental illness affects improvisation remains largely unknown. In this study we focused on socio-motor improvisation in individuals with schizophrenia, one of the more debilitating mental disorder. This represents the ability to improvise gestures during an interaction to promote sustained communication and shared attention. Using a novel paradigm called the mirror game and recently introduced to study joint improvisation, we recorded hand motions of two people mirroring each other. Comparing Schizophrenia patients and healthy controls skills during the game, we found that improvisation was impaired in schizophrenia patients. Patients also exhibited significantly higher difficulties to being synchronized with someone they follow but not when they were leaders of the joint improvisation game. Considering the correlation between socio-motor synchronization and socio-motor improvisation, these results suggest that synchronization does not only promote affiliation but also improvisation, being therefore an interesting key factor to enhance social skills in a clinical context. Moreover, socio-motor improvisation abnormalities were not associated with executive functioning, one traditional underpinning of improvisation. Altogether, our results suggest that even if both mental illness and improvisation differ from normal thinking and behavior, they are not two sides of the same coin, providing a direct evidence that being able to improvise in individual situations is fundamentally different than being able to improvise in a social context.

Influence of perceived emotion and gender on social motor coordination.

Theorists have long postulated that facial properties such as emotion and sex are potent social stimuli that influence how individuals act. Yet extant scientific findings were mainly derived from investigations on the prompt motor response upon the presentation of affective stimuli, which were mostly delivered by means of pictures, videos, or text. A theoretical question remains unaddressed concerning how the perception of emotion and sex would modulate the dynamics of a continuous coordinated behaviour. Conceived in the framework of dynamical approach to interpersonal motor coordination, the present study aimed to address this question by adopting the coupled-oscillators paradigm. Twenty-one participants performed in-phase and anti-phase coordination with two avatars (male and female) displaying three emotional expressions (neutral, happy, and angry) at different frequencies (100% and 150% of the participant's preferred frequency) by executing horizontal rhythmic left-right oscillatory movements. Time to initiate movement (TIM), mean relative phase error (MnRP), and standard deviation of relative phase (SDRP) were calculated as indices of reaction time, deviation from the intended pattern of coordination, and coordination stability, respectively. Results showed that in anti-phase condition at 150% frequency, MnRP was lower with the angry and the female avatar. In addition, coordination was found to be more stable with the male avatar than the female one when both displaying neutral emotion. But the happy female avatar was found to elicit more stable coordination than the neutral female avatar. These results implied that individuals are more relaxed to coordinate with the female than the male, and the sensorimotor system becomes more flexible to coordinate with an angry person. It is also suggested social roles influence how people coordinate, and individuals attend more to interact with a happy female. In sum, the present study evidenced that social perception is embodied in the interactive behaviour during social interaction.

Why you can't be in sync with schizophrenia patients.

Most individuals with schizophrenia will be confronted with some form of stigma. In recent years, clinicians and family members have increasingly contested the term "schizophrenia". Many of them discuss changing this name, as a means to fight stigma. Up until now, surprisingly, most research has been conducted using self-reports and behavioral research is lacking. The aim of our study was to assess through an experimental design if the term "schizophrenia" itself modifies social behaviours. Forty participants were asked to engage in a synchronization task with a dot displayed on a screen and moved by another person. Non-clinicians participants had to synchronize their movements as accurately as possible with either a "schizophrenia" patient, a patient with "neuro-emotional integration disorder" or a "healthy" subject, kept out of sight. Each condition was counterbalanced between participants. In fact, the movements of the dot were pre-recorded (five trajectories) and were therefore identical for all three conditions. Measuring the error between the displayed and performed trajectories, participants exhibited more errors when they thought they were interacting with a patient in comparison to the "healthy" subject. Post-hoc analysis revealed an even higher difference between "schizophrenia" and "healthy" conditions. Altogether, our results show a significant behavioral impact of the term "schizophrenia" with possibly negative consequences on social interactions. The effect of changing the name reduces this impact but remains unclear.

Using mimicry of body movements by a virtual agent to increase synchronization behavior and rapport in individuals with schizophrenia.

Synchronization of behavior such as gestures or postures is assumed to serve crucial functions in social interaction but has been poorly studied to date in schizophrenia. Using a virtual collaborative environment (VCS), we tested 1) whether synchronization of behavior, i.e., the spontaneous initiation of gestures that are congruent with those of an interaction partner, was impaired in individuals with schizophrenia compared with healthy participants; 2) whether mimicry of the patients' body movements by the virtual interaction partner was associated with increased behavioral synchronization and rapport. 19 patients and 19 matched controls interacted with a virtual agent who either mimicked their head and torso movements with a delay varying randomly between 0.5 s and 4 s or did not mimic, and rated feelings of rapport toward the virtual agent after each condition. Both groups exhibited a higher and similar synchronization behavior of the virtual agent forearm movements when they were in the Mimicry condition rather than in the No-mimicry condition. In addition, both groups felt more comfortable with a mimicking virtual agent rather than a virtual agent not mimicking them suggesting that mimicry is able to increase rapport in individuals with schizophrenia. Our results suggest that schizophrenia cannot be considered anymore as a disorder of imitation, particularly as regards behavioral synchronization processes in social interaction contexts.

Standing or swaying to the beat: Discrete auditory rhythms entrain stance and promote postural coordination stability.

Humans seem to take social and behavioral advantages of entraining themselves with discrete auditory rhythms (e.g., dancing, communicating). We investigated the benefits of such an entrainment on posture during standing (spontaneous entrainment) and during a whole-body swaying task (intentional synchronization). We first evaluated how body sway was entrained by different auditory metronome frequencies (0.25, 0.5, and 1.0Hz). We then assessed the stabilizing role of auditory rhythms on postural control, characterized in a dynamical systems perspective by informational anchoring of the head (local stabilization) and fewer transitions from in-phase to anti-phase ankle-hip coordination (global stabilization). Our results revealed in both situations an entrainment of postural movements by external rhythms. This entrainment tended to be more effective when the metronome frequency (0.25Hz) was close to the dominant sway frequency. Particularly, we found during intentional synchronization that head movements were less variable when paced by a slower beat (informational anchoring), and that phase transitions between the two stable patterns in postural dynamics were delayed. Our findings demonstrate that human bipedal posture can be actively or spontaneously modulated by an external discrete auditory rhythm, which might be exploited for the purpose of learning and rehabilitation.

Influence of facial feedback during a cooperative human-robot task in schizophrenia.

Rapid progress in the area of humanoid robots offers tremendous possibilities for investigating and improving social competences in people with social deficits, but remains yet unexplored in schizophrenia. In this study, we examined the influence of social feedbacks elicited by a humanoid robot on motor coordination during a human-robot interaction. Twenty-two schizophrenia patients and twenty-two matched healthy controls underwent a collaborative motor synchrony task with the iCub humanoid robot. Results revealed that positive social feedback had a facilitatory effect on motor coordination in the control participants compared to non-social positive feedback. This facilitatory effect was not present in schizophrenia patients, whose social-motor coordination was similarly impaired in social and non-social feedback conditions. Furthermore, patients' cognitive flexibility impairment and antipsychotic dosing were negatively correlated with patients' ability to synchronize hand movements with iCub. Overall, our findings reveal that patients have marked difficulties to exploit facial social cues elicited by a humanoid robot to modulate their motor coordination during human-robot interaction, partly accounted for by cognitive deficits and medication. This study opens new perspectives for comprehension of social deficits in this mental disorder.

Likability's Effect on Interpersonal Motor Coordination: Exploring Natural Gaze Direction.

Although existing studies indicate a positive effect of interpersonal motor coordination (IMC) on likability, no consensus has been reached as for the effect of likability back onto IMC. The present study specifically investigated the causal effect of likability on IMC and explored, by tracking the natural gaze direction, the possible underlying mechanisms. Twenty-two participants were engaged in an interpersonal finger-tapping task with a confederate in three likability conditions (baseline, likable, and unlikable), while wearing an eye tracker. They had to perform finger tapping at their comfort tempo with the confederate who tapped at the same or 1.5 times of the participant's preferred frequency. Results showed that when tapping at the same frequency, the effect of likability on IMC varied with time. Participants coordinated at a higher level in the baseline condition at the beginning of the coordination task, and a facilitative effect of likability on IMC was revealed in the last session. As a novelty, our results evidenced a positive correlation between IMC and the amount of gaze onto the coordination partner's movement only in the likable condition. No effect of likability was found when the confederate was tapping at 1.5 times of the participant's preferred frequency. Our research suggests that the psychosocial property of the coordinating partner should be taken into consideration when investigating the performance of IMC and that IMC is a parameter that is sensitive to multiple factors.

Interaction patterns and individual dynamics shape the way we move in synchrony.

An important open problem in Human Behaviour is to understand how coordination emerges in human ensembles. This problem has been seldom studied quantitatively in the existing literature, in contrast to situations involving dual interaction. Here we study motor coordination (or synchronisation) in a group of individuals where participants are asked to visually coordinate an oscillatory hand motion. We separately tested two groups of seven participants. We observed that the coordination level of the ensemble depends on group homogeneity, as well as on the pattern of visual couplings (who looked at whom). Despite the complexity of social interactions, we show that networks of coupled heterogeneous oscillators with different structures capture well the group dynamics. Our findings are relevant to any activity requiring the coordination of several people, as in music, sport or at work, and can be extended to account for other perceptual forms of interaction such as sound or feel.

Unravelling socio-motor biomarkers in schizophrenia.

We present novel, low-cost and non-invasive potential diagnostic biomarkers of schizophrenia. They are based on the 'mirror-game', a coordination task in which two partners are asked to mimic each other's hand movements. In particular, we use the patient's solo movement, recorded in the absence of a partner, and motion recorded during interaction with an artificial agent, a computer avatar or a humanoid robot. In order to discriminate between the patients and controls, we employ statistical learning techniques, which we apply to nonverbal synchrony and neuromotor features derived from the participants' movement data. The proposed classifier has 93% accuracy and 100% specificity. Our results provide evidence that statistical learning techniques, nonverbal movement coordination and neuromotor characteristics could form the foundation of decision support tools aiding clinicians in cases of diagnostic uncertainty.

Postural Coordination during Socio-motor Improvisation.

Human interaction often relies on socio-motor improvisation. Creating unprepared movements during social interaction is not a random process but relies on rules of synchronization. These situations do not only involve people to be coordinated, but also require the adjustment of their posture in order to maintain balance and support movements. The present study investigated posture in such a context. More precisely, we first evaluated the impact of amplitude and complexity of arm movements on posture in solo situation. Then, we assessed the impact of interpersonal coordination on posture using the mirror game in which dyads performed improvised and synchronized movements (i.e., duo situation). Posture was measured through ankle-hip coordination in medio-lateral and antero-posterior directions (ML and AP respectively). Our results revealed the spontaneous emergence of in-phase pattern in ML direction and antiphase pattern in AP direction for solo and duo situations. These two patterns respectively refer to the simultaneous flexion/extension of the ankles and the hips in the same or opposite direction. It suggests different functional roles of postural coordination patterns in each direction, with in-phase supporting task performance in ML (dynamical stability) and antiphase supporting postural control in AP (mechanical stability). Although amplitude of movement did not influence posture, movement complexity disturbed postural stability in both directions. Conversely, interpersonal coordination promoted postural stability in ML but not in AP direction. These results are discussed in terms of the difference in coupling strength between ankle-hip coordination and interpersonal coordination.

Humanoid robots versus humans: How is emotional valence of facial expressions recognized by individuals with schizophrenia? An exploratory study.

BACKGROUND: The use of humanoid robots to play a therapeutic role in helping individuals with social disorders such as autism is a newly emerging field, but remains unexplored in schizophrenia. As the ability for robots to convey emotion appear of fundamental importance for human-robot interactions, we aimed to evaluate how schizophrenia patients recognize positive and negative facial emotions displayed by a humanoid robot. METHODS: We included 21 schizophrenia outpatients and 17 healthy participants. In a reaction time task, they were shown photographs of human faces and of a humanoid robot (iCub) expressing either positive or negative emotions, as well as a non-social stimulus. Patients' symptomatology, mind perception, reaction time and number of correct answers were evaluated. RESULTS: Results indicated that patients and controls recognized better and faster the emotional valence of facial expressions expressed by humans than by the robot. Participants were faster when responding to positive compared to negative human faces and inversely were faster for negative compared to positive robot faces. Importantly, participants performed worse when they perceived iCub as being capable of experiencing things (experience subscale of the mind perception questionnaire). In schizophrenia patients, negative correlations emerged between negative symptoms and both robot's and human's negative face accuracy. CONCLUSIONS: Individuals do not respond similarly to human facial emotion and to non-anthropomorphic emotional signals. Humanoid robots have the potential to convey emotions to patients with schizophrenia, but their appearance seems of major importance for human-robot interactions.

Dynamic similarity promotes interpersonal coordination in joint action.

Human movement has been studied for decades, and dynamic laws of motion that are common to all humans have been derived. Yet, every individual moves differently from everyone else (faster/slower, harder/smoother, etc.). We propose here an index of such variability, namely an individual motor signature (IMS) able to capture the subtle differences in the way each of us moves. We show that the IMS of a person is time-invariant and that it significantly differs from those of other individuals. This allows us to quantify the dynamic similarity, a measure of rapport between dynamics of different individuals' movements, and demonstrate that it facilitates coordination during interaction. We use our measure to confirm a key prediction of the theory of similarity that coordination between two individuals performing a joint-action task is higher if their motions share similar dynamic features. Furthermore, we use a virtual avatar driven by an interactive cognitive architecture based on feedback control theory to explore the effects of different kinematic features of the avatar motion on coordination with human players.

Self-Face Recognition in Schizophrenia: An Eye-Tracking Study.

Self-face recognition has been shown to be impaired in schizophrenia (SZ), according to studies using behavioral tasks implicating cognitive demands. Here, we employed an eye-tracking methodology, which is a relevant tool to understand impairments in self-face recognition deficits in SZ because it provides a natural, continuous and online record of face processing. Moreover, it allows collecting the most relevant and informative features each individual looks at during the self-face recognition. These advantages are especially relevant considering the fundamental role played by the patterns of visual exploration on face processing. Thus, this paper aims to investigate self-face recognition deficits in SZ using eye-tracking methodology. Visual scan paths were monitored in 20 patients with SZ and 20 healthy controls. Self, famous, and unknown faces were morphed in steps of 20%. Location, number, and duration of fixations on relevant areas were recorded with an eye-tracking system. Participants performed a passive exploration task (no specific instruction was provided), followed by an active decision making task (individuals were explicitly requested to recognize the different faces). Results showed that patients with SZ had fewer and longer fixations compared to controls. Nevertheless, both groups focused their attention on relevant facial features in a similar way. No significant difference was found between groups when participants were requested to recognize the faces (active task). In conclusion, using an eye tracking methodology and two tasks with low levels of cognitive demands, our results suggest that patients with SZ are able to: (1) explore faces and focus on relevant features of the face in a similar way as controls; and (2) recognize their own face.

The acquisition of socio-motor improvisation in the mirror game.

Socio-motor improvisation is defined as the creative action of two or more people without a script or anticipated preparation. It is evaluated through two main parameters: movement synchronization and movement richness. Experts in art (e.g., dance, theater or music) are known to exhibit higher synchronization and to perform richer movements during interpersonal improvisation, but how these competences evolve over time is largely unknown. In the present study, we investigated whether performing more synchronized and richer movements over time can promote the acquisition of improvisation. Pairs of novice participants were instructed to play an improvisation mirror game in three different sessions. Between sessions, they performed an unintended interpersonal coordination task in which synchronization and richness were manipulated, resulting in four different groups of dyads. Our results demonstrate that synchronization during improvisation improved for all groups whereas movement richness only enhanced for dyads that performed synchronized movements during unintended coordination tasks. Our findings suggest that movement synchrony contributes more than movement richness to the acquisition of socio-motor improvisation in the mirror game.

Further insight into self-face recognition in schizophrenia patients: Why ambiguity matters.

BACKGROUND: Although some studies reported specifically self-face processing deficits in patients with schizophrenia disorder (SZ), it remains unclear whether these deficits rather reflect a more global face processing deficit. Contradictory results are probably due to the different methodologies employed and the lack of control of other confounding factors. Moreover, no study has so far evaluated possible daily life self-face recognition difficulties in SZ. Therefore, our primary objective was to investigate self-face recognition in patients suffering from SZ compared to healthy controls (HC) using an "objective measure" (reaction time and accuracy) and a "subjective measure" (self-report of daily self-face recognition difficulties). METHOD: Twenty-four patients with SZ and 23 HC performed a self-face recognition task and completed a questionnaire evaluating daily difficulties in self-face recognition. Recognition task material consisted in three different faces (the own, a famous and an unknown) being morphed in steps of 20%. RESULTS: Results showed that SZ were overall slower than HC regardless of the face identity, but less accurate only for the faces containing 60%-40% morphing. Moreover, SZ and HC reported a similar amount of daily problems with self/other face recognition. No significant correlations were found between objective and subjective measures (p > 0.05). LIMITATIONS: The small sample size and relatively mild severity of psychopathology does not allow us to generalize our results. CONCLUSIONS: These results suggest that: (1) patients with SZ are as capable of recognizing their own face as HC, although they are susceptible to ambiguity; (2) there are far less self recognition deficits in schizophrenia patients than previously postulated.

Moving attractive virtual agent improves interpersonal coordination stability.

Interpersonal motor coordination is influenced not only by biomechanical factors such as coordination pattern, oscillating frequency, and individual differences, but also by psychosocial factor such as likability and social competences. Based on the social stereotype of "what is beautiful is good", the present study aimed at investigating whether people coordinate differently with physically attractive people compared to less attractive people. 34 participants were engaged in an interpersonal coordination task with different looking (virtual) agents while performing at the same time a reaction time task. Results showed that participants had more stable motor coordination with the moving attractive than with the less attractive agent, and that the difference in motor coordination could not be interpreted by a specific attention allocation strategy. Our findings provide the evidence that physical attractiveness genuinely affects how people interact with another person, and that the temporal-spatial coordinated movement varies with the partner's psychosocial characteristics. The study broadens the perspective of exploring the effect of additional psychosocial factors on social motor coordination.

Social priming enhances interpersonal synchronization and feeling of connectedness towards schizophrenia patients.

What leads healthy individuals to abnormal feelings of contact with schizophrenia patients remains obscure. Despite recent findings that human bonding is an interactive process influenced by coordination dynamics, the spatiotemporal organization of the bodily movements of schizophrenia patients when interacting with other people is poorly understood. Interpersonal motor coordination between dyads of patients (n = 45) or healthy controls (n = 45), and synchronization partners (n = 90), was assessed with a hand-held pendulum task following implicit exposure to pro-social, non-social, or anti-social primes. We evaluated the socio-motor competence and the feeling of connectedness between participants and their synchronization partners with a measure of motor coordination stability. Immediately after the coordination task, all participants were also asked to rate the likeableness of their interacting partner. Our results showed greater stability during interpersonal synchrony in schizophrenia patients who received pro-social priming, inducing in their synchronization partner greater feelings of connectedness towards patients. This greater feeling of connectedness was positively correlated with stronger motor synchronization between participants suggesting that motor coordination partly underlies patients' social interactions and feelings of contact with others. Pro-social priming can have a pervasive effect on abnormal social interactions in schizophrenia patients.

Social priming increases nonverbal expressive behaviors in schizophrenia.

Semantic priming tasks are classically used to influence and implicitly promote target behaviors. Recently, several studies have demonstrated that prosocial semantic priming modulated feelings of social affiliation. The main aim of this study was to determine whether inducing feelings of social affiliation using priming tasks could modulate nonverbal social behaviors in schizophrenia. We used the Scrambled Sentence Task to prime schizophrenia patients according to three priming group conditions: pro-social, non-social or anti-social. Forty-five schizophrenia patients, diagnosed according to DSM-IV-TR, were randomly assigned to one of the three priming groups of 15 participants. We evaluated nonverbal social behaviors using the Motor-Affective subscale of the Motor-Affective-Social-Scale. Results showed that schizophrenia patients with pro-social priming had significantly more nonverbal behaviors than schizophrenia patients with anti-social and non-social priming conditions. Schizophrenia patient behaviors are affected by social priming. Our results have several clinical implications for the rehabilitation of social skills impairments frequently encountered among individuals with schizophrenia.